1. Field of the Invention
The present invention relates to a solid-state imaging apparatus and a signal processing method for transforming image signals output from a honeycomb arrangement to high quality video signals. The apparatus and method of the present invention are advantageously applicable to, e.g., a digital still camera or an image processing apparatus executing signal processing with the image of a subject picked up. More particularly, the present invention is concerned with a digital still camera for converting signals resulting from a shot to image data and recording and/or reproducing the image data, and a recording and/or reproducing method therefor. The digital camera may advantageously be implemented as a digital still camera or a digital video camera.
2. Description of the Background Art
In the imaging art, a solid-state imaging apparatus is extensively used to pick up a desired subject in the form of an image. To meet an increasing demand for a miniature configuration and high resolution, a solid-state imaging device using a single filter type solid photosensitive devices may have the number of photosensitive devices or pixels arranged in, e.g., a square lattice pattern simply increased, as proposed in the early stage of development. However, resolution achievable with this scheme is limited because the light-sensitive area and therefore sensitivity of the individual photosensitive device decreases with an increase in the number of photosensitive devices.
Some different systems for implementing higher resolution from a different point of view are now in study. A first system shifts pixels by using three-filter type photosensitive devices. For example, Shimada et al., “A color camera with CCDs (Charge Coupled Devices) windowed for spatially shifting pixels” is taught in the Journal of the Institute of Image Information and Television Engineers of Japan (former Institute of Television Engineers of Japan), Television and Circuit Group, Reference TBS No. 32-2, pp. 1-19, Feb. 24, 1977. Basically, this color camera uses three color filters for shifting pixels and enhances resolution.
A second system uses line-sequential two-filter photosensitive devices while shifting pixels. Specifically, “A line-sequential 2CCD color camera” is disclosed in Murata et al., the Journal of the Institute of Image Information and Television Engineers of Japan, Technical Report TEBS No. 60-2, pp. 27-32, Jan. 25, 1980). The Technical Report teaches a two-filter imaging system using a G (green) filter and a line-sequential R (red)/B (blue) filter based on an NTSC (National Television System Committee) scheme, a camera arrangement and imaging characteristics for improving resolution in the horizontal direction.
A third system shifts pixels and executes signal processing taking account of shifted pixels, as proposed in, e.g., Japanese patent laid-open publication No. 298275/1995. This document discloses signal processing circuitry for a video camera capable of enhancing resolution in the horizontal and vertical directions by using a G signal derived from a Bayer single-filter system. With this scheme, a sharp image is achievable without limiting a high frequency range at the time of generation of aperture signals.
A fourth system uses a three-filter configuration and feeds the color G to two CCD plates shifted in pixel position from each other. For example, a three-filter, dual green digital camera is disclosed in a paper, Morimmoto et al., “Digital Still Camera Using 3-CCD Image Sensor” , Minolta RD 175 presented at the 70th Fine Imaging Symposium of the Society of Photographic Science and Technology of Japan, pp. 73-76, October, 1995. In this digital camera, the color G is distributed to two of three filters that are shifted in pixel position from each other, and the resulting signals are processed to increase resolution.
Japanese patent laid-open publication No. 340455/1996 proposes, apart from the enhancement of high resolution, an image processing apparatus for feeding signals output from shifted imaging devices to another apparatus so as to cause it to display a faithful image. Specifically, the image processing apparatus processes image data output from photosensitive devices arranged in a non-lattice pattern such that the image data are adaptive to a computer.
A solid-state imaging apparatus including solid-state photosensitive devices each having a unique shape has also been proposed in the past. U.S. Pat. No. 4,441,123, for example, discloses a solid-state imaging apparatus including a filter with a matrix configuration and hexagonal pixels of the same size arranged in correspondence to the matrix of the filter, so that images can be freed from moiré. Japanese Patent laid-open publication No. 77450/1994 teaches solid-state photosensitive devices each having a unique pixel shape and shifted in position from each other. Specifically, each imaging device has a rhombic shape whose sides are inclined by an angle of 45 degrees relative to the vertical direction, and such pixels are shifted from each other. This kind of scheme is directed toward the enhancement of resolution in the vertical direction in relation to an all-pixel read out system.
The above conventional schemes, however, have some problems left unsolved, as follows. The first system executes movie type signal processing for attaching importance to the horizontal resolution. Although this system implements high resolution by shifting pixels, it needs utmost pixel shifting accuracy due to the use of three CCD units. To implement such accuracy, the number of steps of assembling the color camera and therefore the cost of the camera increases.
The second system improves resolution like the first system, but at the sacrifice of the vertical resolution. Further, the movie type vertical resolution is limited by the number of scanning lines, obstructing the application of the system to, e.g., a digital camera using the all-pixel read out system. Moreover, color reproducibility available with the second system is low because the colors of the two filters are complementary to each other. In addition, the second system, like the first system, needs utmost pixel shifting accuracy due to the use of a plurality of filters.
The third system achieves high resolution in both of the horizontal and vertical directions. However, the problem with this system is that because it executes interpolation by detecting correlations in both directions, accurate correlations are not achievable without forcing signal processing circuitry to bear a heavy load. In this sense, accuracy available with the third system is limited.
The fourth system using three filters cannot be easily simplified in configuration, compared to systems using a single filter. Moreover, the highly accurate arrangement matching with the shifted pixels sophisticates a production line.
The image signal processing apparatus of Japanese patent laid-open publication No. 340455/1996 shares the same principle as the third system in that it enhances resolution by signal processing. Shifting pixels in a pattern, as in the above document, is successful to extend the limit of bidimensional visible resolution, as well known in the art. However, because the apparatus shifts pixels while avoiding an increase in the number of pixels, a single picture needs a greater capacity when written to a recording medium in accordance with an increase in the number of pixels. As a result, the number of pictures that can be stored in a recording medium decreases.
The solid-state imaging apparatuses taught in U.S. Pat. No. 4,441,123 and Japanese patent laid-open publication No. 77450/1994 each has a problem that an increase in the number of pixels for implementing high resolution reduces the light-sensitive area and therefore sensitivity of the individual photosensitive device, as stated earlier. Further, decease in the pitch of unit pixels is approaching a limit at which the color aberration of a lens and optical diffraction have adverse influence.
The conventional systems and proposals each increases the amount of data for a picture and thereby reduces the number of pictures that can be stored in a single recording medium.
Now, a digital camera may be constructed with importance attached rather to the number of pictures to be taken than to image quality, depending on the application. For example, a digital camera taught in U.S. Pat. No. 5,034,804 and executing multimode data compression compresses a luminance signal and two chrominance signals subjected to signal processing and writes the compressed signals in a semiconductor memory or similar recording medium. Specifically, the compressed signals or data are written to a recording medium in desired one of a plurality of modes.
Assume that photosensitive devices or cells arranged in shifted positions (or in a honeycomb pattern) are applied to the above multimode compression type digital camera. Then, interpolation based on signals output from the photosensitive devices doubles the amount of data before compression, compared to a square lattice arrangement. By compressing such signals, it is possible to reduce the amount of data to be recorded in accordance with the compression ratio. However, a signal processing time necessary for the digital camera cannot be reduced. The signal processing time is not negligible when it comes to, e.g., a digital camera including more than 1,000,000 photosensitive devices. Particularly, the signal processing time increases an interval between shots when the digital camera is operated in a continuous shoot mode. Furthermore, the camera increases in size and therefore diminishes in value as a product.
Assume image data output from photosensitive devices or pixels simply spatially shifted in position (honeycomb arrangement). The amount of such image data is double the amount of image data output, via apertures arranged in a square lattice pattern, from photosensitive devices whose pixels to be sampled are not spatially offset, as in a monitor for a computer or a television receiver. This also increases the signal processing time.